Chip multiprocessors (CMPs) that include multiple processing cores on a single die can improve system performance. Such CMPs and other multiprocessor systems may be used for highly-threaded (or parallel) applications and to support throughput computing. Particularly in situations where multiple cores and/or threads share one or more levels of a cache hierarchy, difficulties can arise. For example, in some systems, multiple levels of cache memory may be implemented as an inclusive cache hierarchy. In an inclusive cache hierarchy, one of the cache memories (i.e., a lower-level cache memory) may include a subset of data contained in another cache memory (i.e., an upper-level cache memory). Because inclusive cache hierarchies store some common data, eviction of a cache line in one cache level can cause a corresponding cache line eviction in another level of the cache hierarchy to maintain cache coherency. More specifically, an eviction in a higher-level cache causes an eviction in a lower-level cache.
Cache lines in a higher-level cache may be evicted as being stale, although a corresponding copy of that cache line in a lower-level cache may be heavily accessed by an associated core, which may lead to unnecessary cache misses. These cache misses may require significant latencies to obtain valid data from other memory locations, such as a main memory. Thus problems can occur when an inclusive cache hierarchy has a higher-level cache that is shared among multiple processors, for example, multiple cores of a multi-core processor. In this scenario, each core occupies at least some cache lines in the higher-level cache, but all cores contend for the shared resource. When one of the cores uses a small working set which fits inside its lower-level cache, this core rarely (if ever) would have to send requests to the higher-level cache since the requests hit in its lower-level cache. As a result, this core's lines in the higher-level cache may become stale without regard as to how often the core uses them. When sharing the higher-level cache with other cores that continually allocate cache lines into the higher-level cache, this core's data may be evicted, causing performance degradation.
Thus in such a cache hierarchy, fairness and pollution issues can be commonplace due to inter-thread thrashing, high degrees of prefetching, or the existence of streaming data. In addition, when multiple applications are simultaneously running on a multi-core platform, they may have disparate requirements on the cache space and interfere with each other. As one example, the emergence of virtualization as a mechanism to consolidate multiple disparate workloads on the same platform can create cache utilization issues.